Skip to content

[Improvement-18039][Metrics] Add missing metrics for workflow and task state transitions#18038

Open
sanjana2505006 wants to merge 1 commit intoapache:devfrom
sanjana2505006:Improvement-Metrics
Open

[Improvement-18039][Metrics] Add missing metrics for workflow and task state transitions#18038
sanjana2505006 wants to merge 1 commit intoapache:devfrom
sanjana2505006:Improvement-Metrics

Conversation

@sanjana2505006
Copy link
Contributor

@sanjana2505006 sanjana2505006 commented Mar 9, 2026 •
edited
Loading

Purpose

This PR adds missing metrics to the Master module to improve visibility into workflow and task execution states. It addresses the gap where several metrics classes were defined but not utilized in the core execution flow.

Proposed Changes

  • Added workflow instance state tracking in AbstractWorkflowStateAction.
  • Added granular task state tracking (dispatch, success, failure, kill, retry) in AbstractTaskStateAction.
  • Added task timeout metric tracking in TaskTimeoutLifecycleEventHandler.
  • Added workflow instance generation duration measurement in WorkflowExecutionRunnableFactory.

Verification

  • Verified code style with mvn spotless:apply.
  • Performed final code review to ensure metrics integration follows existing patterns.

This PR closes #18039

@github-actions github-actions bot removed the test label Mar 9, 2026
@sanjana2505006 sanjana2505006 changed the title [Improvement-Metrics][master] Add missing metrics for workflow and task state transitions [Improvement][master] Add missing metrics for workflow and task state transitions Mar 9, 2026
@sanjana2505006 sanjana2505006 changed the title [Improvement][master] Add missing metrics for workflow and task state transitions [Improvement] Add missing metrics for workflow and task state transitions Mar 9, 2026
@sanjana2505006 sanjana2505006 changed the title [Improvement] Add missing metrics for workflow and task state transitions [Improvement][master]Add missing metrics for workflow and task state transitions Mar 9, 2026
@sanjana2505006 sanjana2505006 changed the title [Improvement][master]Add missing metrics for workflow and task state transitions [Improvement][master] Add missing metrics for workflow and task state transitions Mar 9, 2026
SbloodyS
SbloodyS reviewed Mar 9, 2026
View reviewed changes
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the pull request notice and create an issue first. @sanjana2505006

@sanjana2505006 sanjana2505006 changed the title [Improvement][master] Add missing metrics for workflow and task state transitions [Improvement-18039][Metrics] Add missing metrics for workflow and task state transitions Mar 9, 2026
@sanjana2505006
Copy link
Contributor Author

sanjana2505006 commented Mar 9, 2026

Thank you @SbloodyS , for the guidance! I've created the issue #18039 and updated the PR title and commit to reference it.

@sanjana2505006 sanjana2505006 requested a review from SbloodyS March 9, 2026 06:06
@SbloodyS SbloodyS added the improvement make more easy to user or prompt friendly label Mar 9, 2026
@SbloodyS SbloodyS added this to the 3.4.2 milestone Mar 9, 2026
@SbloodyS
Copy link
Member

SbloodyS commented Mar 10, 2026

Please check the failed CI. @sanjana2505006

@sanjana2505006
Copy link
Contributor Author

sanjana2505006 commented Mar 10, 2026 •
edited
Loading

Sure, @SbloodyS, I have an exam today. I’ll take a look at the failed CI once I’m done 😃...

@sanjana2505006
Copy link
Contributor Author

sanjana2505006 commented Mar 10, 2026

Hello @SbloodyS, I've updated the PR to address the CI failures. Please have a look when you have time and share your feedback. Thank you!

@github-actions github-actions bot added the test label Mar 11, 2026
@SbloodyS
Copy link
Member

SbloodyS commented Mar 16, 2026

UT still failed. @sanjana2505006

ruanwenjun
ruanwenjun reviewed Mar 20, 2026
View reviewed changes
Copy link
Member

@ruanwenjun ruanwenjun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your PR.
It might be better to add event metrics first.

WorkflowEventBusFireWorker#doFireSingleWorkflowEventBus

We may need to provide a public method to handle changes in instance state, and then use listeners or similar mechanisms to update the metrics.
So it’s not suitable to add this directly at the moment, but I’ll be doing that soon.

...nscheduler/server/master/engine/task/lifecycle/handler/TaskTimeoutLifecycleEventHandler.java Outdated
// The task instance is not active, means it is already finished.
return;
}
org.apache.dolphinscheduler.server.master.metrics.TaskMetrics.incTaskInstanceByState("timeout");
Copy link
Member

@ruanwenjun ruanwenjun Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
org.apache.dolphinscheduler.server.master.metrics.TaskMetrics.incTaskInstanceByState("timeout");
TaskMetrics.incTaskInstanceByState("timeout");

Copy link
Contributor Author

@sanjana2505006 sanjana2505006 Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed this entirely.
And have deleted this direct metric call and moved the timeout tracking logic to the event bus worker instead

.../apache/dolphinscheduler/server/master/engine/task/statemachine/AbstractTaskStateAction.java Outdated
taskInstance.setState(TaskExecutionStatus.FAILURE);
taskInstance.setEndTime(taskFatalEvent.getEndTime());
taskInstanceDao.updateById(taskInstance);
TaskMetrics.incTaskInstanceByState("fail");
Copy link
Member

@ruanwenjun ruanwenjun Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
TaskMetrics.incTaskInstanceByState("fail");
TaskMetrics.incTaskInstanceByState(TaskExecutionStatus.FAILURE);

Copy link
Contributor Author

@sanjana2505006 sanjana2505006 Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these direct metric calls inside the state actions have been deleted. The failure tracking is now natively handled by the event bus worker alongside the other task events.

@sanjana2505006
[Improvement-18039][Metrics] Add missing metrics for workflow and tas…
e30a3d3 
…k state transitions

This PR adds missing metrics for task and workflow execution into DolphinScheduler metrics. It also adopts an event-driven mechanism to track Task metrics cleanly upon Task event bus fires.
@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 25, 2026

Quality Gate Failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 60%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@caishunfeng caishunfeng Awaiting requested review from caishunfeng caishunfeng is a code owner

@SbloodyS SbloodyS Awaiting requested review from SbloodyS SbloodyS is a code owner

@ruanwenjun ruanwenjun Awaiting requested review from ruanwenjun ruanwenjun is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@sanjana2505006 sanjana2505006

Labels

backend improvement make more easy to user or prompt friendly test

Projects

None yet

Milestone

3.4.2

Development

Successfully merging this pull request may close these issues.

[Improvement][Metrics] Add missing metrics to Master and Worker modules

3 participants

@sanjana2505006 @SbloodyS @ruanwenjun